Parallel Implementation of Partially Connected Recurrent Networks
نویسندگان
چکیده
| Recurrent neural networks are suitable for solving problems with temporal extent e.g. speech recognition, time series prediction, sequence generation. The biggest problem is, however, its computational complexity during the training process. Using the well-known Real Time Recurrent Learning rule by D. Zipser and R.J. William 2], the training time of each epoch is of order O(n 4) where n is the total number of hidden units and output units. To cope with this low eeciency, we have devised a new network model called the partially connected recurrent network 3]. Using the modiied learning rule, the computational complexity of training becomes only O(mn + np), where m is number of outputs and p is the number of external inputs. In additions this new recurrent model is very suitable for running on a regularly connected SIMD parallel computer since each neuron needs only to communicate with those connected to it directly. We have veriied this idea by implementing both a ring-structured recurrent network and a grid-structured recurrent network on the DECmpp 12000Sx System with 8192 processors. We can show that the training time per epoch will now increase only linearly, i.e. O(m+n+p), which is a big advantage for solving large scale problems.
منابع مشابه
Connection Reduction of the Recurrent Networks
There are many variations on the topology of recurrent networks. Models with fully-connected recurrent weights may not be superior to those models with sparsely connected recurrent weights in terms of capacity, time for training and generalization ability. In this paper, we show that the fully-connected recurrent networks can be reduced to functionally equivalent partially-connected recurrent n...
متن کاملFault Detection and Location in DC Microgrids by Recurrent Neural Networks and Decision Tree Classifier
Microgrids have played an important role in distribution networks during recent years. DC microgrids are very popular among researchers because of their benefits. Protection is one of the significant challenges in the way of microgrids progress. As a result, in this paper, a fault detection and location scheme for DC microgrids is proposed. Due to advances in Artificial Intelligence (AI) and s...
متن کاملNavigation of a Mobile Robot Using Virtual Potential Field and Artificial Neural Network
Mobile robot navigation is one of the basic problems in robotics. In this paper, a new approach is proposed for autonomous mobile robot navigation in an unknown environment. The proposed approach is based on learning virtual parallel paths that propel the mobile robot toward the track using a multi-layer, feed-forward neural network. For training, a human operator navigates the mobile robot in ...
متن کاملForward kinematic analysis of planar parallel robots using a neural network-based approach optimized by machine learning
The forward kinematic problem of parallel robots is always considered as a challenge in the field of parallel robots due to the obtained nonlinear system of equations. In this paper, the forward kinematic problem of planar parallel robots in their workspace is investigated using a neural network based approach. In order to increase the accuracy of this method, the workspace of the parallel robo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1994